Scanning environments and tracking unmanned aerial vehicles

ABSTRACT

Systems and methods for scanning environments and tracking unmanned aerial vehicles within the scanned environments are disclosed. A method in accordance with a particular embodiment includes using a rangefinder off-board an unmanned air vehicle (UAV) to identify points in a region. The method can further include forming a computer-based map of the region with the points and using the rangefinder and a camera to locate the UAV as it moves in the region. The location of the UAV can be compared with locations on the computer-based map and, based upon the comparison, the method can include transmitting guidance information to the UAV. In a further particular embodiment, two-dimensional imaging data is used in addition to the rangefinder data to provide color information to points in the region.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/127,476, filed on Mar. 3, 2015, titled SCANNING ENVIRONMENTS ANDTRACKING UNMANNED AERIAL VEHICLES, which is herein incorporated byreference in its entirety.

TECHNICAL FIELD

The present disclosure is directed generally to scanning environmentsand tracking unmanned aerial vehicles.

BACKGROUND

Unmanned aerial vehicles (UAVs) have become increasingly popular devicesfor carrying out a wide variety of tasks that would otherwise beperformed with manned aircraft or satellites. Such tasks includesurveillance tasks, imaging tasks, and payload delivery tasks. However,UAVs have a number of drawbacks. For example, it can be difficult tooperate UAVs, particularly autonomously, in close quarters, e.g., nearbuildings, trees, or other objects. In particular, it can be difficultto prevent the UAVs from colliding with such objects. While techniquesexist for controlling UAVs and other autonomous vehicles in complexenvironments, such techniques typically rely on hardware that is tooheavy, too bulky, and/or requires too much power to be carried by atypical UAV without dramatically impacting the range and/or payloadcapacity of the UAV. Accordingly, there remains a need for techniquesand associated systems that can allow UAVs to safely and accuratelynavigate within their working environments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1C are partially schematic, block diagrams illustrating anoverall process for scanning an environment and tracking a UAV withinthe environment, in accordance with an embodiment of the presenttechnology.

FIG. 2 is a flow diagram of a representative scanning and trackingprocess configured in accordance with a particular embodiment of thepresent technology.

FIG. 3 illustrates a representative scanner configured in accordancewith an embodiment of the present technology.

FIG. 4 is a block diagram illustrating components of a representativescanner operating in a scanning mode, in accordance with arepresentative embodiment of the present technology.

FIG. 5 is a block diagram illustrating components of a representativescanner operating in a tracking mode, in accordance with an embodimentof the present technology.

FIGS. 6A and 6B are schematic illustrations a UAV having a codingarrangement to facilitate identifying and/or tracking the UAV inaccordance with an embodiment of the present technology.

FIG. 7 illustrates an arrangement of multiple scanners positioned toscan an environment in accordance with an embodiment of the presenttechnology.

FIG. 8 illustrates an arrangement for carrying out a scanning operationvia equipment carried by a UAV in accordance with an embodiment of thepresent technology.

FIG. 9A schematically illustrates a conventional process for scanning anobject.

FIG. 9B schematically illustrates a technique for selectively scanningfeatures of an object in accordance with an embodiment of the presenttechnology.

FIG. 10 is a schematic block diagram illustrating a process foridentifying and scheduling features to be scanned in accordance with anembodiment of the present technology.

FIG. 11 is a flow diagram illustrating a process for planning andexecuting a flight path trajectory in accordance with an embodiment ofthe present technology.

FIG. 12 illustrates a scanner positioned in an environment in accordancewith an embodiment of the present technology.

FIGS. 13A-13B illustrate isometric and plan views, respectively, of animage of the environment shown in FIG. 12.

FIG. 14 illustrates an image of another portion of the environment shownin FIG. 12.

FIGS. 15A-15C illustrate front, top, and side views of an environmentthat includes wind turbine, and an associated UAV flight path, inaccordance with an embodiment of the present technology.

FIGS. 16A-16D illustrate a process for obtaining data from wind turbineblades while the blades are in motion, in accordance an embodiment ofthe present technology.

DETAILED DESCRIPTION

The present technology is directed generally to unmanned aerial vehicles(UAVs) and more particularly, to systems and methods for scanning theenvironments in which unmanned aerial vehicles operate, planning aflight for the unmanned aerial vehicles through one or more of theenvironments, and tracking the motion of the unmanned aerial vehicles inthose environments. Specific details of several embodiments of thedisclosed technology are described below with reference to particular,representative configurations. In other embodiments, the disclosedtechnology may be practiced in accordance with UAVs and associatedsystems having other configurations. In still further embodiments,particular aspects of the disclosed technology may be practiced in thecontext of autonomous vehicles other than UAVs (e.g., autonomous cars orwatercraft). Specific details describing structures or processes thatare well-known and often associated with UAVs, but that mayunnecessarily obscure some significant aspects of the presentlydisclosed technology, are not set forth in the following description forpurposes of clarity. Moreover, although the following disclosure setsforth several embodiments of different aspects of the disclosedtechnology, several other embodiments of the technology can haveconfigurations and/or components different than those described in thissection. As such, the present technology may have other embodiments withadditional elements and/or without several of the elements describedbelow with reference to FIGS. 1A-15C.

Several embodiments of the disclosed technology may take the form ofcomputer-executable instructions, including routines executed by aprogrammable computer or controller. Those skilled in the relevant artwill appreciate that the technology can be practiced on computer orcontroller systems other than those shown and described below. Thetechnology can be embodied in a special-purpose computer, controller, ordata processor that is specifically programmed, configured orconstructed to perform one or more of the computer-executableinstructions described below. Accordingly, the terms “computer” and“controller” as generally used herein include a suitable data processorand can include internet appliances and hand-held devices, includingpalm-top computers, wearable computers, cellular or mobile phones,multi-processor systems, processor-based or programmable consumerelectronics, network computers, laptop computers, mini-computers and thelike. Information handled by these computers can be presented at anysuitable display medium, including a liquid crystal display (LCD). As isknown in the art, these computers and controllers commonly have variousprocessors, memories (e.g., non-transitory computer-readable media),input/output devices, etc.

The present technology can also be practiced in distributedenvironments, where tasks or modules are performed by remote processingdevices that are linked through a communications network. In adistributed computing environment, program modules or subroutines may belocated in local and remote memory storage devices. Aspects of thetechnology described below may be stored or distributed oncomputer-readable media, including magnetic or optically readable orremovable computer discs, as well as distributed electronically overnetworks. Data structures and transmissions of data particular toaspects of the technology are also encompassed within the scope of thepresent technology.

1. Overview

FIGS. 1A-1C schematically identify several functions that can beperformed as part of the technology disclosed in the presentapplication. FIG. 1A illustrates a process for scanning an environment190 in which a UAV is expected to operate. The environment 190 can beindoors and/or outdoors, and include one or more objects 191, forexample, a building. A system 100 in accordance with an embodiment ofthe present technology includes a scanner 110 that is used to scan theenvironment 190 and, in particular, the object or objects 191, toproduce a computer-based model 136 of the environment 190. The scanner110 can be controlled by a controller 130. The same controller 130 canbe used to perform a variety of additional functions, also describedherein, in accordance with particular embodiments of the presenttechnology. In other embodiments, these functions may be divided overmultiple controllers. Multiple controllers may be referred to herein as“controllers,” and one or more controllers can be included in a “controlsystem.” As discussed above, the controller 130 can have any of avariety of suitable configurations, e.g., a laptop computerconfiguration.

As shown schematically in FIG. 1 B, the controller 130 can include aprocessor 131, a memory 132 (e.g., non-transitory computer-readablemedia), and one or more input/output devices 133. The input/outputdevices 133 can include an input device 134 (e.g., a keyboard) and anoutput device 135 (e.g., a screen displaying a graphical userinterface). As shown in FIG. 1 B, the output device 135 is displaying acomputer-based object image 136 of the environment 190, including theobject 191 shown in FIG. 1A. The output device 135 also displays aproposed flight path image 137, representing a proposed flight path fora UAV in the region of the object 191. Accordingly, the controller 130can be used to plan the UAV's flight path, e.g., by identifyingobstacles in the flight path, alerting the operator to such obstacles,and/or suggesting or automatically implementing alternate flight paths.

A further aspect of the system is illustrated in FIG. 1C. The controller130 and the scanner 110, initially described above with reference toFIG. 1A, can be used to track the motion of a UAV 150 as it operates inthe environment 190 and around the object 191. In particular, thescanner 110 can be used in this context to identify the actual locationof the UAV 150 relative to the computer-based model of the object 191,and can communicate this information to the UAV 150 via a communicationdevice 138 (e.g., an antenna). Accordingly, the system 100 can be usedto provide real-time data regarding the UAV's position and, by comparingthat data to the model described above with reference to FIGS. 1A and1B, can provide positional feedback to the UAV. The feedback can be usedto adjust the UAV's actual flight path 151, as necessary, in comparisonto the planned flight path 137 to allow the UAV to carry out its missionwith a high degree of precision and/or avoid collisions.

Thus, the system 100 can be used to (a) map the environment in which theUAV operates, (b) facilitate defining a flight plan for the UAV in theenvironment, and (c) provide feedback to the UAV if it deviates from theflight plan. Further details of each of these functions are describedbelow with reference to FIGS. 2-15C.

2. Scanning and Tracking

FIG. 2 is a flow diagram illustrating a representative process forscanning an environment and tracking a UAV within the environment, inaccordance with an embodiment of the present technology. The overallprocess can include the subprocesses of scanning, planning a flightpath, and tracking the UAV. Depending on the embodiment, thesesubprocesses can be performed sequentially as part of a single overallprocess, or any of the subprocesses can be performed individually. Theprocess 200 can include scanning the local environment with a ground orother off-board scanner (block 201). In a representative embodiment, thescanner is located on the ground, e.g., supported by a tripod or anothersuitable structure. In other embodiments, the scanner can have otherlocations, e.g., it can be carried aloft by a balloon or aircraft.

In block 202, the process includes planning a flight path within theenvironment, where the flight path may be manually input by a humanoperator or calculated automatically based on various constraints. Indecision block 203, the process includes determining whether the plannedpath is achievable, for example, determining if the path is free ofobstacles. If not, the process returns to block 202 for an update to theplanned flight path. In various embodiments, the process can excludedetected obstacles when computing a revised path. If the path isachievable (e.g., is free of obstacles given the proximity constraintsand dynamics of the UAV), the process proceeds to block 204, whichbegins the tracking portion of the process. As part of this process, thesystem acquires a track on the UAV as it is in use, e.g., as it takesoff and flies. When the tracking information is acquired (e.g., when thelocation of the UAV is acquired with a camera and/or rangefinder), thislocation information can be transmitted to the controller 130 thatcontrols the tracking device in pan and tilt axes, continuously keepingthe UAV within the field of view. The controller 130 launches the UAV(block 205) and directs the UAV to the beginning of its planned flightpath (block 206). In block 207, the UAV follows the flight path to itsconclusion, and at block 208 lands at the end of the mission. During thecourse of the flight, the information received via tracking the UAV(e.g., the coordinates of the UAV) can be transmitted to the controller130 to update the trajectory of the UAV. For example, if the UAV beginsdeviating from the pre-defined flight plan, the information can be usedto guide (e.g., automatically) the UAV back to the planned route.Accordingly, portions of the controller can be located on-board the UAV,off-board the UAV, or both, depending on the manner in which UAV controland feedback are implemented.

As the UAV flies (blocks 205-208), the process includes checking, e.g.,continually checking, to determine whether the UAV is successfully beingtracked (block 209). As long as the UAV is being successfully tracked,the process continues as indicated. If the system loses track of theUAV, then in block 210, the process includes taking actions tofacilitate re-acquiring the track on the UAV. Such steps can includedirecting the UAV to hover, land, and/or fly toward the scanner, and/orperforming a sweeping (and/or other) motion with the scanner to locatethe UAV, and/or other suitable actions.

As part of the process of flying the UAV, the operator can manipulatethe progress of the UAV along the flight path. For example, the operatorcan use controls similar to VCR controls to fast-forward, reverse, orpause the progress of the UAV along the flight path. Such controls, ofcourse, will be dependent on the UAV configuration, e.g., the UAV musthave a hover capability in order for it to be paused. A similar approachcan be used to preview the flight of the UAV along a proposed flightpath, as part of the planning process for the flight path, which isdescribed further below with reference to FIGS. 11-15C.

FIG. 3 is an isometric illustration of a representative scanner 110having features configured in accordance with an embodiment of thepresent technology. The scanner 110 can include a support 111 thatcarries a camera 117 and a rangefinder 118. The camera 117 can beconfigured to produce two-dimensional optical images of the environmentaround the scanner 110 by receiving radiation from the environment inthe visible spectrum, infrared range, and/or other suitable frequencyranges. The rangefinder 118 can include an emitter 119 and a receiver120. The emitter 119 emits a signal that reflects from an object in theenvironment and is received by the receiver 120. The distance from thescanner 110 to the object is then determined or estimated by using anyof a variety of suitable techniques, including estimating the amount oftime required for the signal to transit from the emitter 119 to theobject and back to the receiver 120 (“time of flight”). Accordingly, thecamera 117 can identify and transmit two-dimensional information aboutthe environment, and the rangefinder 118 can add the third dimension.

The camera 117 and the rangefinder 118 can be carried by a tilt stage114 and can be moved together as a unit to scan the environment aroundthe scanner 110. The tilt stage 114 carries a tilt motor 115 thatrotates the camera 117 and the rangefinder 118, as a unit, about a firstaxis (e.g., a horizontal axis H). A corresponding tilt encoder 116tracks the motion of the camera 117 and the rangefinder 118 relative tothe horizontal axis H. A pan motor 112 carried by the support 111rotates the tilt stage 114 (including the camera 117 and the rangefinder118) as a unit about a second axis (e.g., a vertical axis V). A panencoder 113 tracks the rotational position of the camera 117 and therangefinder 118 around the vertical axis V. Accordingly, the pan motor112 and the tilt motor 115 can rotate to the camera 117 and therangefinder 118 through arcs sufficient to cover a roughly hemisphericalvolume around the scanner 110.

In a particular embodiment, the rangefinder 118 can include a lidardetector, which emits and receives laser light (e.g., IR laser light).Suitable lidar detectors have range capabilities in the hundreds ofmeters, depending on factors that include the size of the emitter 119and receiver or detector 120, and the ranging technology used. Asdiscussed above the ranging technology can include a time of flighttechnique in one embodiment. In other embodiments, other techniques,such as SETS techniques, can produce suitable results without requiringdirect time of flight calculations, at lower cost and lower (but stillsuitable) resolution. The scans can be conducted in a methodical sweeppattern, or a coarse scan followed by a detailed scan, or an adaptivescan (described further below with reference to FIGS. 9A-10), or viaanother suitable technique. In other embodiments, the rangefinder 118can emit signals other than a laser signal, suitable for detecting thedistance between the scanner 110 and the objects around it. For example,a radar signal can be used for tracking, though it is expected that alaser signal will out-perform a radar signal for scanning. In anotherembodiment, the laser scanner can be replaced by multiplehigh-resolution cameras or a structured light arrangement to perform thescanning process.

FIG. 4 is a schematic block diagram illustrating the system 100operating in a scanning mode, in accordance with an embodiment of thepresent technology. As shown in FIG. 4, the camera 117 records images ofthe surrounding environment 190, while the rangefinder 118 records thedistances to the objects within the environment 190. The camera 117accordingly generates RGB data (or other optical data) 123 and therangefinder 118 generates depth or distance data 124. The RGB data 123is transmitted to the processor 131. The depth data 124 is convertedfrom spherical coordinates to Cartesian coordinates at the processorusing conventional converter logic 125 that operates on information fromthe tilt encoder 116 and the pan encoder 113. This transformedcoordinate information is used to generate a point cloud from the datacaptured by the camera 117 and the rangefinder 118 using the point cloudbuilder logic 127.

The system 100 can also include a motor control unit that providesinstructions to the tilt motor 115 and the pan motor 112, and is itselfunder the control of a scan controller 430. The scan controller 430 caninclude scan region logic 122, which is described in further detailbelow with reference to FIGS. 9A-10. The scan controller 430 can be astand-alone controller, or it can be integrated with one or more othercontrollers, e.g., the controller 130 described above with reference toFIGS. 1A-1C.

A power supply 126 provides power to the various components of thesystem 100. The input/output device 133 receives information from anoperator and provides output information to the operator. The result ofthe process performed during the scanning mode shown in FIG. 4 is acomputer-based model (e.g., a point cloud) of the environment 190.

In particular embodiments, the data obtained during the scanningoperation and used to build the three-dimensional model can besupplemented with additional data. For example, the model can beaugmented or enhanced with photographs or other sensor readings taken bythe scanner camera 117 or by a UAV that flies through the scannedenvironment. This operation can be conducted in real time in someembodiments, and offline in others. The enhancement can include addingthe color information contained in the camera image to the points in thepoint cloud to produce a more realistic, colored model displaying thespectral representation at each point.

The data obtained from a particular scan can be stored and used later bythe same scanner (e.g., in a track mode), or by a different scanner,also in the track mode, or by a UAV operating independently of anoffboard tracking device. In a particular embodiment, when a tracker ispositioned in a particular area, it can automatically access prior scansmade of that environment, and those scans can be downloaded to thescanner as it operates in the track mode. Furthermore, a particular scanmay be used at multiple locations, not necessarily the same as thelocation at which the initial scan was made. For example, a scan made ofa particular wind turbine, power line tower, or other structure can beused to survey or inspect other structures that have an identicalconfiguration, for example, wind turbines of the same make and model. Insuch cases, the scanned information can be calibrated to the particularlocation of an individual wind turbine, which may be different for eachwind turbine.

FIG. 5 illustrates the system 100 operating in a tracking mode. In thismode, the camera 117 identifies a UAV 150 (e.g. by tracking a fiducialsuch as an LED carried by the UAV, by matching airframe characteristicsas trained by a machine learning algorithm, or by other means) andgenerates two-dimensional tracking data 123. Simultaneously, therangefinder 118 generates range data 561 corresponding to the distancebetween the scanner 110 and the UAV 150, as the UAV flies. Thisinformation is provided to the processor 131 which generates an estimateof the position of the UAV 150 using position estimator logic 562. Theposition information can be transmitted to the I/O device 133, which cantransmit the information via an information signal 563 to the UAV 150.With this information, the UAV 150 can modify, adjust, and/or compensatefor variations in the flight path it executes. The position informationis also transmitted to the Motor Control Unit 121 which actuates the panmotor 112 and tilt motor 115 to continuously keep the UAV 150 within thefield of view of the camera 117 and in front of the rangefinder 118 asthe UAV flies.

In another embodiment, the system can track the UAV 150 by combiningcorner cube interferometry with a laser-based position sensing device togenerate the position estimate for the UAV 150. The position informationcan be transmitted to the I/O device 133 and UAV 150 so that it canmodify, adjust, and/or compensate for variations in its flight path.

In particular embodiments, the system can, in addition to transmittinginformation to the UAV 150, receive information from the UAV 150. In aparticular aspect of this embodiment, the information received from theUAV 150 can correspond to an identity of the UAV 150 and/or anorientation of the UAV 150. FIGS. 6A and 6B illustrate a representativeUAV 650 having a body 652 and one or more rotors 653 (four are shown inFIGS. 6A and 6B) for propulsion. The UAV 650 can also include a physicalencoding schema, for example a ring 654 or lighting system that in turnincludes one or more coding patterns 655 (FIG. 6B). The coding patterns655 can be used to identify the type of UAV detected by the scanner 110(FIG. 5) during its tracking process, for example, to confirm that theproper UAV is being tracked. This procedure can also be used to identifynew or additional UAVs that may enter the environment in which thetracking process is being conducted. Accordingly, the coding patterns655 can perform the function of an identifying fiducial.

In addition, the coding patterns 655 can have different characteristicsat different circumferential positions around the coding ring 654.Accordingly, when the system 100 detects the coding patterns 655, it candetect the orientation of the UAV 650. This information can in turn beused to help control, guide, and/or direct the UAV 650. To obtain thisinformation, the camera 117 (or a separate, dedicated camera) can obtainan image of the coding patterns, which is then processed (e.g., in themanner of a bar coder). In another embodiment, the system 100 can issuean inquiry signal 664 (e.g., in addition to the information signal 563)that impinges on and is reflected by the coding pattern 655. Thereflected signal is then processed to determine the orientation and/oridentity of the UAV 650. In both of the foregoing embodiments, thecoding pattern 655 is passive. In other embodiments, the coding pattern655 can be actively transmitted by the UAV 650, e.g., via a radioemission, an LED pattern or flash sequence, and/or other suitablearrangements.

An advantage of the orientation function described above is that it maybe more accurate than the orientation function provided by an on-boardinertial navigation system. Another advantage is that on-board inertialnavigation systems typically include a magnetometer, which can beaffected by nearby structures, e.g., magnetic or metal structures. Theoff-board scanner can provide orientation information that is unaffectedby such structures, allowing the UAV to operate in environments it mightnot otherwise be able to.

In at least some instances, a single scanner may be insufficient toprovide all the data required for the UAV to carry out its tasks. Forexample, as shown in FIG. 7, a target object 791 may include a buildinghaving target features that are not accessible to a single scanner at asingle location. In one embodiment, the scanner 110 is moved amongmultiple locations to provide sufficient data for the desired pointcloud. At each location, the scanner 110 obtains two-dimensional imagedata and depth data, as discussed above, and the multiple data sets canbe “stitched” together to form a composite computer model. In otherembodiments, multiple scanners 110 can be used to both scan the featuresof the building, and provide tracking for the UAV 150 as it flies itsmission. The scanners 110 can communicate with each other, as indicatedby arrows M so as to hand off tracking responsibility from one scannerto the other as the UAV 150 conducts its mission.

In yet another embodiment, illustrated in FIG. 8, the scanning functioncan be performed by a UAV 850 itself. In such an embodiment, thescanning equipment is made compact and lightweight enough to be carriedby the UAV 850. In some embodiments, the scanning equipment may becompromised in some manner to reduce its weight. For example, thescanning equipment may have a coarser resolution, reduced range, reducedfield of view and/or other weight-saving features. However, with thesystem on board, the UAV 850 can assume the positions identified byletters A, B, C and D to map the features of the building 791. Aftermapping the features, the UAV 850 can carry out its mission, asindicated by letter E.

In the foregoing embodiment, the tracking function described above withreference to FIG. 5 may be performed by comparing images taken by theUAV 850 while it is in flight, with point cloud information produced byscans taken by the UAV while it is on the ground at positions A-D. Inanother embodiment, the tracking operation is carried out by two UAVs850, each equipped with a scanner. Accordingly, one UAV 850 can fly amission (e.g., an inspection mission) and the second UAV 850 can trackthe first UAV 850. The tracking function can be performed from any ofthe scanning positions A-D described above, or (with accommodations foran offset) from a different location.

The system can perform still further functions directed to tracking theUAVs. For example, in the track mode, the system can scan theenvironment to determine if a UAV is in the area. Upon detecting theUAV, the tracker can use any of the arrangements described above toidentify the UAV and, optionally, its orientation. The tracker can alsocommunicate with UAVs that are out of visual range or beyond line ofsight (BLOS). Such UAVs can relay their position and trajectory to thetracker (for example, via GPS and radio communication) and the trackercan accordingly orient itself toward the position where the UAV isexpected to come within range. Once the UAV comes within range, thetracker can carry out the tracking operations described above. In otherembodiments, the UAV can be tracked using ADS-B in addition to, or inlieu of GPS. Still further, the system can scan a pre-scannedenvironment to check for new objects that may have entered theenvironment after the initial scan was completed. Such new objects caninclude a UAV, or other objects that may represent obstacles to a UAV.If such objects are detected, the three-dimensional model of theenvironment can be updated to reflect the existence of those objects.

As noted above with reference to FIG. 7, some arrangements can includemultiple scanners for performing scanning and/or tracking functions. Insuch cases, these scanners can communicate with each other to providecontinuity within the three-dimensional models that they create, and totransmit information related to UAVs in their individual environments.For example, the scanners can cooperate to give control priority to thescanner that is closest to the UAV, or that is projected to have thelongest contact with the UAV before a handoff. In addition, whether anoperation includes one scanner or more than one scanner the system canschedule on-board photo/recording/sensing functions. For example, it maybe advantageous to direct the UAV to perform a sensing function from aprecise location. When the scanner is performing in the tracking mode,it can alert the UAV when the UAV reaches the precise location, in realtime, to ensure that the sensing function is performed at the properlocation. This technique can also be used to capture an image of the UAVwhen it is in a particular location, using the ground-based cameraportion of the scanner. In this manner, the UAV can capture an image ata particular location, and the scanner can capture an image of the UAVat that location. Both techniques can be used to improve thereproducibility of data, e.g., if the same image is to be obtained atdifferent times. The UAV position information from the tracker can beused to reduce computation and improve the fit of photos to the cloud ofpoints. For example, the downstream analysis of the data can be simplerand/or more accurate when the data are tagged with the precise locationfrom which the data were taken.

3. Adaptive Scanning

FIG. 9A illustrates an object 991 being scanned by a scanner 110 inaccordance with a conventional scanning technique. Using this technique,the scanner 110 methodically and sequentially pans and tilts to producescan paths 994 a (shown in dotted lines) that cover the object 991. Thisprocess can be time-consuming, particularly when only certain featuresof the object 991 may be of real interest to the end user. For example,the end user may be more interested in certain target features 992. Thetarget features 992 can include straight lines that form target regions993, for example, the roof and windows of the building.

FIG. 9B illustrates the scanner 110 carrying out an adaptive scanroutine in accordance with an embodiment of the present technology. Inthis embodiment, the scan paths 994 b executed by the scanner 110 aredirected specifically to the target features 992 and therefore thetarget regions 993. As a result, the scanner 110 spends less timeunnecessarily scanning features that are not of interest. Accordingly,the process of scanning the object 991 can take significantly less timethan it does using conventional techniques, and can use significantlyfewer resources.

FIG. 10 is a block diagram illustrating a process for carrying out thescanning operation shown in FIG. 9B. An overall process 1000 inaccordance with an embodiment of the present disclosure includes animage processing block 1001 that in turn includes an image acquisitionblock 1002. At block 1002, the process includes acquiring image data,e.g., from the camera 117. At block 1003, the image data are analyzed,for example to identify particular features, and more particularly,features that are likely to be of interest, e.g., by an end user. Suchfeatures can include lines, corners, and/or other target shapes, orfeatures that may be identified by a user.

At block 1004, the target shapes are grouped into regions, which maycorrespond to doors, windows, and/or other shapes or areas. Block 1005and 1006 include classifying each of the target regions, for example, byselecting regions that are of interest for targeted scanning by therangefinder 118 (block 1009). This process can be performed manually,automatically, or via a combination of manual and automated techniques.For example, the process can include using computer-based shaperecognition routines to identify target features (or target regions) anda user can approve or reject the automatically generated selections. Inother embodiments, the user can assume full control of the selectionprocess (fully manual) or cede control (fully automated). In particularembodiments, algorithms such as Hough transforms are used to identifyfeatures and/or regions of interest. In other embodiments, machinelearning and/or neural network techniques can be used to identifyfeatures and/or regions of interest, thus reducing the required userinput. If the features and/or regions are of interest, a scheduler 1007collects the information for each region and identifies an optimal (orat least expedient) scan path (block 1008). At block 1009, the scancontroller 430 (FIG. 4) directs the camera 117 and the associatedrangefinder 118 (FIG. 4) along the scan paths identified at block 1008.The output (block 1010) is a three-dimensional model, which the UAV usesto navigate and perform its mission.

4. Flight Path Planning

FIGS. 11-15C illustrate representative processes for forming or definingflight paths in accordance with which the UAV conducts its mission inthe environment scanned by the scanner, as described above. Arepresentative process 1100 shown in FIG. 11 includes scanning theenvironment or importing a three-dimensional model of the environment(block 1101). At block 1102, the process includes constructing a flightpath within the environment. In a particular embodiment, the flight pathis constructed using user input 1103. For example, the user can view thethree-dimensional model on a computer screen and, via a mouse, keys, ajoystick, and/or other input devices, construct the flight path. Becausethe flight path is three-dimensional, the system can allow the user torotate the image in any manner (e.g., tumble the image) so as to viewthe environment from multiple perspectives and provide a suitablethree-dimensional flight path.

In another embodiment, the flight path can be automatically generated,as indicated by block 1104. In this case, the process may requireadditional inputs 1105. Such inputs can include a generalcharacterization of the desired flight path, for example, anidentification of the regions of interest and the distance away from theregions of interest that the UAV is to fly. It may be of particularimportance to maintain a set distance from an object, (a) to preventcollisions with the object, and/or (b) to keep the object within thetarget range of a sensor aboard the UAV. Additionally, the informationcan include the speed of the UAV, the flight dynamics and capabilitiesof the UAV, and any restrictions or limitations associated with theability of the UAV to point its camera or other sensor toward the targetregion. Based on this information, and an input from the user (e.g., theuser can click on a feature or object, and select an offset distance),the system can automatically generate a flight path. The flight path canbe a single-pass path or a multi-pass path (e.g., a “lawnmower” pattern)depending on factors including the field of view of the UAV-basedsensor, and the desired resolution of the object it senses.

Block 1106 includes calculating a predicted trajectory based on the UAVpath determined at block 1102, taking into account the speed of the UAV,the flight dynamics and capabilities of the UAV, and other restrictionsas described above. Block 1107 includes checking to determine if thepath is free of collisions, for example, collisions with the objectsrepresented in the model imported at block 1101. If the path doesinclude collisions (or greater than a threshold likelihood ofcollisions), the process returns to block 1102 for adjustments to theflight path. As an example, the process can prevent areas of the paththat are causing the collisions from being included in a subsequentconstruction of a suitable path. If not, then the flight path can beinitiated and executed, as indicated at block 1108.

FIG. 12 illustrates the scanner 110 in a representative environment 1290that includes first objects (e.g., statues 1291 a) and second objects(e.g., buildings 1291 b). FIG. 13A illustrates the computer-based modelof the environment, including the statues 1291 a and the buildings 1291b, based on data obtained from the scanner 110. FIG. 13A alsoillustrates a proposed flight path 1337 that may be constructed by auser viewing the pointcloud of the environment 1336. FIG. 13B is a planview (top-down) of the pointcloud environment 1336 shown in FIG. 13A.The user can toggle between preset views (e.g., plan view, side view,front view, etc.), or can move gradually and incrementally at any pacefrom one view to another by tumbling around the pointcloud environment1336.

FIG. 14 illustrates one of the buildings 1291 b shown in FIGS. 13A and13B along with the proposed flight path 1337. The proposed flight path1337 includes an interfering portion 1337 a where the UAV would collidewith, or come too close to, the building 1291 b. Accordingly, the useris alerted to the interference, for example, by displaying theinterfering portion 1337 a of the flight path in a different color thanthe non-interfering portion. In other embodiments, the system can useother alerting techniques (e.g., flashing lights, and/or an audiblesignal). Additionally, the system can prevent the user from flying themission until the interfering portion is removed, e.g. by modifying theflight path.

FIGS. 15A, 15B, and 15C are front, top, and side views of an object 1591that includes a wind turbine 1591. FIGS. 15A-C also illustrate a flightpath 1551 along which a UAV 150 flies to examine the wind turbine 1591,and a (schematic) field of view 1593 for the ground-based scanner. Thethree-view illustration is suitable for indicating to the user that nocollisions or likely collisions exist for the UAV 150. As discussedabove, the image may also be tumbled or otherwise rotated to providethis information to the user. In any of the foregoing embodiments, thedata collected by the scanner can be used to aid the UAV in performingat least two functions. First, the data can be used to aid the UAV incarrying out the substance of its mission. For example, as shown inFIGS. 15A-C, the mission may include inspecting the blades of a windturbine. Accordingly, the image data can provide an accurate indicationof the location of the blades and/or other areas of interest that theUAV may focus on as part of its mission. Second, the data can aid theUAV in avoiding obstacles as it carries out its mission. For example,the image data can allow the UAV to avoid striking the pylon 1592 thatcarries the wind turbine blades.

In the embodiment shown in FIGS. 15A-15C, the wind turbine 1591 isstopped while the UAV 150 performs its inspection. In anotherembodiment, the UAV 150 can perform the inspection while the turbineblades are spinning, as illustrated in a representative embodiment shownin FIGS. 16A-16D.

Beginning with FIG. 16A, the wind turbine 1591 includes the pylon 1592described above, which carries three blades 1595. When the blades 1595are in motion, they sweep out a disk D. Each blade 1595 can includemultiple regions extending inwardly from the blade tips. Tworepresentative regions are illustrated in FIGS. 16A-16D as a firstregion 1596 a and a second region 1596 b. In the embodiment shown inFIG. 16A, the UAV 150 moves to a first position P1 located toward theouter periphery of the disk D, so as to be aligned with the firstregions 1596 a of each blade 1595 as the blades pass by.

Referring now to FIG. 16B, the first region 1596 a of one of the blades1595 has rotated into the field of view 1597 of a sensor carried by theUAV 150. With the first region 1596 a in the field of view 1597, asdetermined by the UAV and/or the ground-based scanner/tracker, thesensor is activated. For example, when the sensor includes a camera, thecamera is activated to take an image of the first region 1596 a. The UAV150 can remain in the first position P1 and, as the first region 1596 aof each successive blade 1595 passes into the field of view 1597, theUAV 150 can activate the sensor to take an image or collect other data.When the first regions 1596 a of each blade 1595 have been imaged orotherwise sensed, the UAV 150 can move to a second position P2, as shownin FIG. 16C. From the second position P2, the UAV 150 can image orotherwise sense the second regions 1596 b of each blade 1595, as shownin FIG. 16D. The foregoing process can be repeated for additionalregions of interest, e.g., regions located inwardly from the secondregion 1596 b toward the hub 1599 from which the blades 1595 extend. Anadvantage of the foregoing approach is that the wind turbine 1591 neednot be halted while the UAV 150 obtains diagnostic information,including but not limited to image data of the blades 1595. This in turncan increase the run time of the wind turbine 1591 without compromisingon the need for obtaining periodic diagnostic data.

One feature of at least some of the foregoing embodiments is that thesystem can include a combination of a camera (or other two-dimensionalimaging device) and a laser scanner (or other rangefinder) to map out anenvironment in which a UAV is expected to fly. One advantage of thisarrangement is that it is relatively inexpensive when compared totraditional surveying laser scanners, or spinning lidar devices. Anotheradvantage of the foregoing arrangement is that it provides capabilitiesdifferent than those of existing systems. For example, existingsurveying rangefinders operate at about 0.5-1 Hz and provide resolutionat a millimeter or sub-millimeter level. Embodiments of the presentsystem operate at rates of 50 Hz-100 Hz, which (unlike a 1 Hz rate) aresufficient to keep pace with the flight speeds and offset distances oftypical UAVs. The resolution of embodiments of the present system can bereduced to the centimeter level, which is sufficient for typical UAVtasks, and reduces the cost of the system.

Yet another expected advantage of embodiments of the disclosed systemsis that they can be located off-board the UAV and therefore do notrequire the UAV to provide payload capacity and processing power for thescanning and mapping equipment. Instead, the payload capacity can bereserved for mission-performance equipment, and the power can beconserved for conducting the UAV's mission. This feature can beparticularly important for UAVs that have limited battery capacity.Still another advantage is that the combined scanner and rangefinderarrangement is compact and is therefore more portable than other mappingequipment, for example, equipment that includes an array of infraredcameras.

Another feature of at least some of the foregoing embodiments is thatthe scanner can perform a tracking function, in addition to a scanningfunction. An advantage of this feature is that the UAV can be told whatits position is from an off-board source on roughly the same time scaleat which it is flying. In particular, the UAV can receive updatesmultiple times per second, and at a rate that allows it to change courseif necessary, for example, to avoid obstacles. Another advantage of thisfeature is that the UAV's position can be continually compared to apre-defined flight path that has been defined using an accurate,three-dimensional model of the local environment. Accordingly, thelikelihood for the UAV to get off track is reduced, and if the UAV doesget off track, the system can redirect the UAV with a high degree ofaccuracy.

Yet another feature of at least some of the foregoing embodiments isthat the process for planning a flight can be automated, semi-automated,and/or more visually interactive than conventional waypoint planningarrangements. An advantage of this arrangement is that it can speed upthe process for flightpath planning, particularly, in cluttered localenvironments. Another advantage is that the flight path can be moreaccurate, which again has particular utility in GPS-denied or crowdedareas where GPS is either unavailable or insufficient to provide thelevel of detail required to navigate through such areas. Yet anotheradvantage is that the interface can be intuitive. This is so for atleast the reason that the objects presented at the interface arevisually representative of the objects as they actually appear. Abuilding looks like a building, a tree looks like a tree, etc. Stillanother advantage is that the interface can better simulate (viarotation or tumbling) the actual three-dimensional environment, and theuser can appreciate the three-dimensional nature of the flightenvironment by rotating the image so as to see obstacles and targets ofinterest from multiple angles. This is unlike typical flight planningarrangements in which the user has to imagine the height of obstacles.Still another advantage is that at least some aspects of the flightplanning process can be automated. For example, the user can input arepresentative flight path, which can be automatically evaluated forcollision problems. The flight path can also automatically be adjustedto provide a constant offset from a particular object, for example, awind turbine blade that is being inspected by the UAV flying along theflight path.

Still another feature of at least some of the foregoing embodiments isthat the process for obtaining the data used to build thethree-dimensional model can be adapted to account for the environmentthat is scanned. In particular, the scanning process can be focused onareas of interest, rather than the environment as a whole. An advantageof this approach is that the process for scanning the environment can befaster because areas that are similar (e.g., large flat surfaces) andareas that are physically unscannable (e.g., the sky) may be avoided andtherefore do not take time to map. Another advantage is that the limitednumber of data points available for a particular model can be focused onthe areas of interest so that those areas are modeled with greaterresolution than areas that are of less importance. In addition, theoverall number of points required to define a particular model can bereduced. Still further, the demands on (and therefore the cost of) thescanning motors are reduced when fewer data points are taken. Thisapproach can reduce the cost/complexity of the overall system.

5. Further Embodiments

In particular embodiments, the scanner 110 remains in the same positionfor both the scanning operation and the tracking operation. In otherembodiments, the scanner may be moved between these two operations. Insuch cases, the process for tracking the UAV can include accounting foran offset between the position at which the scanner conducted the scan,and the position from which the scanner tracks the UAV. The offset canbe determined using GPS and/or other suitable techniques. A particularscan may be tagged with its GPS coordinates so that it can be used formultiple missions, each of which may have the tracker at a differentlocation. The GPS coordinates from which the initial scan is made canprovide a constant reference point suitable for multiple uses.Alternatively, a local coordinate system can be used in lieu of GPS.

In particular embodiments, the fiducial carried by the UAV can take theform of a coding pattern, as described above. In other embodiments, thefiducial can include other arrangements, for example, an LED or LEDpattern. In still further embodiments, when in the track mode, thescanner can identify the UAV by matching certain characteristics of theUAV (e.g., its silhouette) with a library of pre-identified silhouettesas trained by machine learning algorithms, e.g. a deep neural network.

From the foregoing, it will be appreciated that specific embodiments ofthe present technology have been described herein for purposes ofillustration, but that various modifications may be made withoutdeviating from the technology. For example, while certain embodimentshave been described in the context of UAVs, in other embodiments,similar techniques may be used for other types of robots, e.g.,water-based robots and in particular, submersible robots, which canconfront issues associated with three-dimensional environments similarto those confronted by air vehicles. In particular embodiments describedabove, the tracking mode includes tracking coordinates. In otherembodiments, the tracking mode can include tracking the direction oftravel, the UAV speed, and/or other variables, in addition to or in lieuof tracking the vehicle coordinates. The tracking process and theassociated scanning process can be performed outdoors, as shown inseveral of the Figures above, and can also be performed indoors. Thesystem can cycle or oscillate between the scanning mode and the trackingmode, even when used in the same environment, for example, to detectchanges in the environment. Such changes can then be imported into thepoint cloud or other computer-based image file so as to update theassociated three-dimensional model. Particular embodiments of the systemwere described above in the context of a quad-rotor UAV, and in otherembodiments, the UAV can have other quad-rotor or non-quad-rotorconfigurations.

The scanner can have different arrangements in different embodiments.For example, in a representative embodiment described above, the cameraand rangefinder each have independent optical paths to the environmentand the UAV. In another embodiment, a prism or other beam splitterprovides identical optical information to both the camera and therangefinder. This approach can improve the correspondence betweeninformation provided by the camera and information provided by therangefinder. In other embodiments, the scanner can include devices otherthan a single camera and a single rangefinder, e.g., multiple cameras(having different fields of view, wavelength sensitivities, and/or otherfunctional parameters) and/or multiple rangefinders (having differentsensitivities over different distances, and/or other functionalparameters).

Certain aspects of the technology described in the context of aparticular embodiment may be combined or eliminated in otherembodiments. For example, in particular embodiments, the flight pathplanning algorithm described above can be used independently of thescanning and tracking processes described above. In another embodiment,the adaptive scanning techniques described above can be usedindependently of the tracking and/or flight planning processes describedabove. Further, while advantages associated with certain embodiments ofthe technology have been described in the context of those embodiments,other embodiments may also exhibit such advantages, and not allembodiments need necessarily exhibit such advantages to fall within thescope of the present technology. Accordingly, the present disclosure andassociated technology can encompass other embodiments not expresslyshown or described herein.

The present application is related to the following co-pending,concurrently-filed applications, each of which is herein incorporated byreference in its entirety: U.S. patent application Ser. No.15/059,030,filed Mar. 2, 2016, titled SCANNING ENVIRONMENTS AND TRACKINGUNMANNED AERIAL VEHICLES, U.S. patent application Ser. No. 15/059,069,filed Mar. 2, 2016, titled SCANNING ENVIRONMENTS AND TRACKING UNMANNEDAERIAL VEHICLES, and PCT Application No. PCT/US16/20509,filed Mar. 2,2016, titled SCANNING ENVIRONMENTS AND TRACKING UNMANNED AERIALVEHICLES. To the extent any materials incorporated herein by referenceconflict with the present disclosure, the present disclosure controls.

The invention claimed is:
 1. A method, performed by a computing systemhaving a memory and a processor, for tracking a flight of an unmannedaerial vehicle in an environment using a three-dimensional model of theenvironment, the method comprising: with a scanner system, comprising atleast one camera that is remote from the unmanned aerial vehicle,tracking the flight of the unmanned aerial vehicle in the environmentbased at least in part on: identifying the unmanned aerial vehiclewithin at least one image captured by the at least one camera,determining a position and/or orientation of the unmanned aerial vehiclebased at least in part on at least one physical coding patternassociated with the unmanned aerial vehicle, confirming an identity ofthe unmanned aerial vehicle based at least in part on the at least onephysical coding pattern associated with the unmanned aerial vehicle, andcausing at least a rotation of the at least one camera so that theunmanned aerial vehicle remains within a field of view of the at leastone camera, and comparing the flight of the unmanned aerial vehicle inthe environment to a flight path constructed in the three-dimensionalmodel; determining that the flight of the unmanned aerial vehicle hasdeviated from the flight path constructed in the three-dimensionalmodel; and in response to determining that the flight of the unmannedaerial vehicle has deviated from the flight path constructed in thethree-dimensional model, redirecting the flight of the unmanned aerialvehicle in accordance with the flight path constructed in thethree-dimensional model at least in part by transmitting guidanceinformation to the unmanned aerial vehicle.
 2. The method of claim 1,wherein tracking the flight of the unmanned aerial vehicle in theenvironment comprises: repeatedly capturing an image with at least onecamera; and analyzing the captured images to estimate the position andorientation of the unmanned aerial vehicle.
 3. The method of claim 1,wherein tracking the flight of the unmanned aerial vehicle in theenvironment comprises: repeatedly capturing an image with at least onecamera; analyzing the captured images to locate the unmanned aerialvehicle; and determining or estimating a distance to the unmanned aerialvehicle with at least one rangefinder wherein the at least onerangefinder comprises at least one of: a light ranging rangefinder, or aradio frequency ranging rangefinder.
 4. The method of claim 1, whereintracking the flight of the unmanned aerial vehicle in the environmentcomprises: receiving Automatic Dependent Surveillance—Broadcastinformation from the unmanned aerial vehicle.
 5. The method of claim 1,further comprising: while tracking the unmanned aerial vehicle, updatingthe three-dimensional model of the environment.
 6. The method of claim5, wherein updating the three-dimensional model of the environmentcomprises: capturing, with the at least one camera, images of theenvironment; identifying features within one or more of the capturedimages of the environment; generating, with a rangefinder, depth data atleast in part by determining the distance from the rangefinder to eachof a plurality of the identified features; transforming the generateddepth data from spherical coordinates to Cartesian coordinates based atleast in part on information collected from a tilt encoder and a panencoder; and generating, with the processor, an updatedthree-dimensional model of the environment based at least in part on thetransformed depth data.
 7. The method of claim 6, further comprising:comparing the updated three-dimensional model of the environment to thethree-dimensional model of the environment to identify new objectswithin the environment.
 8. The method of claim 1, further comprising:providing, for presentation to a user, information characterizing avisual representation of the three-dimensional model of the environment.9. The method of claim 1, wherein comparing the actual flight of theunmanned aerial vehicle in the environment to the flight pathconstructed in the three-dimensional model comprises: receiving speedand location information from the unmanned aerial vehicle; and comparingthe received speed and location information from the unmanned aerialvehicle to the flight path constructed in the three-dimensional model.10. The method of claim 1, further comprising: receiving, from a user,input related to a proposed flight path for the unmanned aerial vehiclethrough the environment; and constructing the flight path in thethree-dimensional model based at least in part on the received input andflight dynamics and capabilities of the unmanned aerial vehicle.
 11. Themethod of claim 1, further comprising: retrieving the three-dimensionalmodel from a library of previously-generated three-dimensional models.12. The method of claim 11, wherein at least one of thethree-dimensional models of the library of previously-generatedthree-dimensional models is used to track the path of an unmannedvehicle through multiple environments.
 13. The method of claim 1,wherein the at least one captured image of the unmanned aerial vehicleis captured by a camera attached to another unmanned aerial vehicle. 14.The method of claim 1, wherein the three-dimensional model of theenvironment is generated at least in part by: capturing, with each of aplurality of scanners located at different locations, images of theenvironment and depth information associated with each of a plurality offeatures identified in the images, wherein each scanner comprises atleast one camera and at least one rangefinder; and based on the depthinformation, stitching together at least two of the images of theenvironment to form a composite three-dimensional model of theenvironment.
 15. The method of claim 1, wherein tracking the flight ofthe unmanned aerial vehicle in the environment comprises: comparingimages taken by a camera attached to the unmanned aerial vehicle to thethree-dimensional model of the environment.
 16. The method of claim 1,further comprising: receiving information corresponding to restrictionson a flight path for the unmanned aerial vehicle; and providing feedbackif the flight path constructed in the three-dimensional model violatesthe restrictions.
 17. The method of claim 1, wherein the physical codingpattern includes at least one of an LED pattern or flash sequence. 18.The method of claim 1, wherein the physical coding pattern is passive.19. The method of claim 1, wherein the physical coding pattern isactively transmitted.
 20. A non-transitory computer-readable storagemedium storing instructions that, if executed by a computing systemhaving a processor, cause the computing system to perform a method fortracking a flight of an unmanned aerial vehicle in an environment usinga three-dimensional model of the environment, the method comprising:tracking the flight of the unmanned aerial vehicle in the environmentbased at least in part on: identifying the unmanned aerial vehiclewithin at least one image captured by at least one camera remote fromthe unmanned aerial vehicle, determining a position and/or orientationof the unmanned aerial vehicle based at least in part on at least onephysical coding pattern associated with the unmanned aerial vehicle,confirming an identity of the unmanned aerial vehicle based at least inpart on the at least one physical coding pattern associated with theunmanned aerial vehicle, and causing at least a rotation of the at leastone camera so that the unmanned aerial vehicle remains within a field ofview of the at least one camera, and comparing the actual flight of theunmanned aerial vehicle in the environment to a flight path constructedin a three-dimensional model; determining that the actual flight of theunmanned aerial vehicle has deviated from the flight path constructed inthe three-dimensional model; and in response to determining that theactual flight of the unmanned aerial vehicle has deviated from theflight path constructed in the three-dimensional model constructed inthe three-dimensional model, transmitting guidance information to theunmanned aerial vehicle.
 21. The computer-readable storage medium ofclaim 20, wherein the tracking of the flight of the unmanned aerialvehicle in the environment is further based at least in part on positioninformation received from the unmanned aerial vehicle.
 22. A systemcomprising: a scanner comprising: a camera, a rangefinder, a tiltencoder, and a pan encoder; and one or more processors that, whenexecuting instructions stored on at least a memory, are configured to:generate a three-dimensional model of an environment based at least inpart on: information collected from the camera, information collectedfrom the rangefinder, information collected from the tilt encoder, andinformation collected from the pan encoder; calculate a projected flightpath for an unmanned aerial vehicle through three-dimensional model ofthe environment, wherein the scanner is remote from the unmanned aerialvehicle; cause the unmanned aerial vehicle to begin a flight through theenvironment in accordance with the projected flight path; track theflight of the unmanned aerial vehicle through the environment based atleast in part on: information collected from the camera, informationcollected from the tilt encoder, information collected from the panencoder, identifying the unmanned aerial vehicle within at least oneimage captured by the camera, determining a position and/or orientationof the unmanned aerial vehicle based at least in part on at least onephysical coding pattern associated with the unmanned aerial vehicle,confirming an identity of the unmanned aerial vehicle based at least inpart on the at least one physical coding pattern associated with theunmanned aerial vehicle, and causing at least a rotation of the cameraso that the unmanned aerial vehicle remains within a field of view ofthe camera, and compare the tracked flight of the unmanned aerialvehicle through environment to the projected flight path of the unmannedaerial vehicle through the three-dimensional model of the environment.23. The system of claim 22, wherein the one or more processors arefurther configured to: receive an indication of a plurality of targetfeatures of the environment, wherein the camera captures only images ofthe environment that include one or more of the target features of theenvironment.